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Abstract 

Background: Despite great need, the number of HIV prevention implementation studies remains limited. The 
challenge for researchers, in this time of limited HIV services agency resources, is to conceptualize and test how to 
disseminate efficacious, practical, and sustainable prevention programs more rapidly, and to understand how to do 
so in the absence of additional agency resources. We tested whether training and technical assistance (TA) in a 
couple-based HIV prevention program using a Web-based modality would yield greater program adoption of the 
program compared to training and TA in the same program in a manual-based modality among facilitators who 
delivered the interventions at 80 agencies in New York State. 

Methods: This study used a cluster randomized controlled design. Participants were HIV services agencies (N = 80) 
and up to 6 staff members at each agency (N = 253). Agencies were recruited, matched on key variables, and 
randomly assigned to two conditions. Staff members participated in a four-day, face-to-face training session, 
followed by TA calls at two and four months, and follow-up assessments at 6, 12, and 18 months post- training and 
TA. The primary outcomes examined number of couples with whom staff implemented the program, mean number 
of sessions implemented, whether staff implemented at least one session or whether staff implemented a complete 
intervention (all six sessions) of the program. Outcomes were measured at both the agency and participant level. 

Results: Over 18 months following training and TA, at least one participant from 13 (33%) Web-based assigned 
agencies and 19 (48%) traditional agencies reported program use. Longitudinal multilevel analysis found no 
differences between groups on any outcomes at the agency or participant level with one exception: Web-based 
agencies implemented the program with 35% fewer couples compared with staff at manual-based agencies 
(IRR 0.35, CI, 0.13-0.94). 

Conclusion: Greater implementation of a Web-based program may require more resources and staff exposure, 
especially when paired with a couple-based modality. Manual-based and traditional programs may hold some 
advantage or ease for implementation, particularly at a time of low economic resources. 

Trial registration: ClinicalTrials.gov identifier: NCT01 863537 
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Background 

Despite the need for implementation science in HIV 
prevention in the United States, the number of imple- 
mentation studies remains limited [1-4]. This may be 
due to the fact that while there is urgency for HIV scien- 
tists to identify, demonstrate and promote robust imple- 
mentation models, the prevention field is experiencing a 
shift in scientific priorities [5,6]. Shifts include move- 
ment from behavioral towards biomedical prevention ap- 
proaches, promotion of behavioral couple-based models 
to overcome barriers in traditional, individually-based 
models, a recognition that efficacious multisession pro- 
grams need to be briefer for sustainability in service set- 
tings [7] and that programming should target individuals 
living with HIV and their intimate partners [6]. Historic- 
ally, the U.S. is also slowly rebounding from an eco- 
nomic recession. This context, in light of the traditional 
barriers to implementation efforts in real world HIV ser- 
vices agencies, may be holding back more rapid gains in 
HIV prevention implementation science in the U.S. 

The challenge for researchers, in this time of limited 
HIV services agency resources, is to conceptualize and 
test how to more rapidly disseminate cost-effective, effi- 
cacious, practical, and sustainable prevention programs- 
including behavioral approaches [8]. We cannot abandon 
behavioral approaches because even as biomedical innova- 
tions for HIV prevention show great promise in efficacy 
trials, they likely cannot be fulfilled without behavioral in- 
terventions to support their adoption and dissemination 
[9]. Barriers evident in individual level prevention ap- 
proaches have led to increased testing of couple-based 
strategies which are often found to be more efficacious in 
promoting HIV counseling and testing [10,11] and sup- 
porting medication adherence [12], but which may re- 
quire more complex implementation strategies [13,14]. 
Given the promise of couple-focused approaches (e.g., bal- 
ancing attention to the dyad, positive reinforcement for 
relationship-based behaviors, conflict management) [15], 
scientists have called for improved efforts at examining 
their dissemination [13]. Yet, couple-based services pre- 
sent a unique set of implementation challenges [13,14]. 
How can we best balance complexity with ease of access 
or more rapid dissemination? 

Dissemination of evidence-based HIV prevention in 
the U.S. has taken place largely through the Centers for 
Disease Control and Prevention's (CDC) Diffusion of 
Evidence-Based Interventions (DEBI) program in which 
agencies receive funding and technical assistance (TA) 
to deliver 'packaged' programs that had been selected as 
'best evidence' [16,17]. This national approach is based 
on best evidence from Kelly et al. [18,19] and offers 
paper-based manuals, training workshops, and follow-up 
TA. TA may be formally or informally provided, and in- 
cludes advice, assistance and training pertaining to the 



implementation and maintenance of the program. Since 
then there are a number of studies which examine the 
conduct and monitoring processes of agencies funded to 
implement CDC DEBI packages [20-24], but few have 
examined actual implementation of any DEBI program 
[1,25] and none have tested alternative, scalable ap- 
proaches to disseminating these packaged interventions. 
At the same time, computer-assisted and Internet-based 
prevention programs have demonstrated better short- 
and long-term efficacy for a range of outcomes [26-28] 
compared to manually based approaches, including HIV 
prevention risk reduction [29,30]. Advantages of Web- 
based technologies are that they may expedite the use of 
effective interventions in real- world settings [30,31] by of- 
fering the potential for time efficiency, cost-effectiveness, 
scalability, and enhanced learning via Internet-based ap- 
proaches [32-35]. Scientists are calling for experiments 
testing computer and Internet-based dissemination for- 
mats on implementation [7,19,30,36-41]. 

This clinical trial tested the adoption of an HIV pre- 
vention intervention model to clinic-based health ser- 
vices in neighborhoods throughout New York State. The 
goal of the study was to examine whether using a Web- 
based approach for implementation of a couple-focused 
HIV prevention program, Connect, yields greater imple- 
mentation than a traditional, manual-based approach 
used by facilitators at HIV service agencies. We tested 
implementation of couple-based intervention modalities 
(manual versus Web-based) into real world settings. The 
study investigated how the Internet may be used to sup- 
port the delivery of a behavioral 'packaged' manual-based 
intervention and 'packaged' computer based intervention 
in agency settings [30]. Although both manual-based and 
Web-based approaches require a facilitator for imple- 
mentation, we hypothesized that the Web-based approach 
might ease the implementation process, leading to in- 
creased use. Compared to manual-based approaches, 
Web-based program activities may offer greater ease 
of access to program and re-training materials, increase 
the number of facilitators who could train on and use the 
program following an initial training, and increase re- 
hearsal and self-efficacy for implementing activities among 
those facilitators who use the program [42]. We had four 
sets of hypotheses regarding whether more agencies (or 
participants) assigned to the Web-based program would 
implement the program [e.g., delivered sessions). We 
hypothesized that more agencies (or participants) as- 
signed to implement the Web-based program would 
report Hi, delivering Connect with more couples; H 2 , 
delivering more Connect sessions; H 3 , delivering at least 
one Connect session; and H 4 , delivering at least one 
complete Connect intervention (all six sessions) over 
the 18 months of follow up, compared to agencies (and 
staff) assigned to the manual-based approach. In addition 
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we observed post-training use of TA and barriers to 
implementation. 

Methods 

Study design 

This study was a longitudinal matched pairs cluster ran- 
domized trial conducted between 2007 and 2012 [43,44]. 
We recruited agencies providing HIV services in New 
York State. Eligible agencies had 'not-for-profit' status in 
the U.S., provided HIV prevention services to heterosex- 
ual men and women, and agreed to send at least one 
participating staff member to receive training on one of 
two versions of the Connect program (traditional or 
Multimedia). We screened 145 agencies identified from 
a list of state-funded community service providers (CSPs) 
and multi-service agencies (MSAs) as well as non-state 
funded agencies identified through five Web sites of HIV 
services coalitions. The final sample was made up of 80 
agencies (see Figure 1). 

HIV services agencies are highly variable in terms of 
their size, scope of service, and available financial re- 
sources. Given this variability, and the relatively small 
sample size, we decided that a matched pair approach 
would best guard against imbalance across arms after 
random assignment. We identified two factors that the 
research team, in consultation with our community 



advisory board (CAB), believed might most strongly 
influence the primary outcomes: number of full and 
part time paid staff providing HIV prevention services in 
the prior year; and number of clients receiving multi- 
session HIV prevention services in the prior year. To cre- 
ate matches for pairing [43,44], the project statistician 
rank ordered and categorized these two factors (in ascend- 
ing order) from agency level screening assessments (com- 
pleted by agency Directors) to create a two-way table with 
an even-number frequency in each cell. The statistician 
then generated a random number for each case: standard 
normal (Gaussian) random variates. He then sorted by 
random numbers in each cell and assigned every two cases 
as one pair in order. The first case was assigned to the 
Multimedia condition; the second to the Traditional con- 
dition as described below. 

Intervention 

Manual-based intervention package 

Participants from agencies randomized to the manual- 
based intervention were invited to a four-day, face-to- 
face structured orientation and training on the original, 
manualized Connect program. In addition, we offered 
planned, investigative-team-initiated telephone consulta- 
tions to provide TA at two and four months following 
the workshop. The traditional Connect program and 



Agencies assessed for eligibility 

(n=145) 



Excluded (n=65) 

♦ Not meeting inclusion criteria (n=23) 

♦ Declined to participate (n=25) 

♦ Other reasons (n= 17) 



Randomized 

(n=80 agencies; 253 staff) 

I 



Multimedia (Web-based) 

(n=40 agencies; 122 staff) 

► Received allocated intervention (n=40 agencies) 

► Received allocated intervention (n=1 19 staff) 



Traditional (Manual-based) 

(n=40 agencies; 131 staff) 

♦ Received allocated intervention (n=40 agencies) 

♦ Received allocated intervention (n=126 staff) 



6 month follow-up 

40, (100%) agencies; 108 (90.7%) staff) 
12 month follow-up 

40, (100%) agencies; 107 (89.9%) staff) 
18 month follow-up 

38, (95%) agencies; 105 (88.2%) staff) 



6 month follow-up 

40, (100%) agencies; 120 (95.2%) staff) 
12 month follow-up 

40, (100%) agencies; 115 (91.2%) staff) 
18 month follow-up 

39, (98%) agencies; 106 (84.1%) staff) 



Figure 1 Study design flowchart of intervention testing Web-based versus manual-based implementation of a couple-focused HIV 
prevention intervention. 
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facilitator training curriculum is one of the CDC DEBIs 
(see http://www.effectiveinterventions.org/en/HighImpact 
Prevention/Interventions/Connect.aspx) [43,44] and is 
provided in six weekly, two-hour sessions by a single 
facilitator to a couple using a standard sequence of 
guided activities. Connect is theoretically and empiric- 
ally derived, grounded in participatory research [45], 
with demonstrated efficacy to reduce risk among HIV- 
negative, HIV-positive or serodiscordant heterosexual 
couples [46-50]. Connect focuses on the relationship as 
the target of change; its core elements include redefining 
risk from individual protection to preserving the relation- 
ship; examining issues of fidelity, gender differences, 
power, and decision making; and using modeling, role- 
play, and feedback to promote mastery of skills that 
enhance risk reduction behavior, including couple commu- 
nication, negotiation, problem solving, and social-support 
enhancement. Connect has been adapted and replicated 
with efficacy in a number of trials domestically and glo- 
bally with youth, men who have sex with men (MSM) 
couples, drug-affected, and HIV serodiscordant African 
American couples [51-55]. 

Web-based intervention package 

Participants from agencies randomized to the Web-based 
intervention were invited to a four-day, face-to-face 
structured orientation and training on the 'Multimedia 
Connect' program (translated from the original into a 
Web-based format). In addition, we offered planned, 
investigative-team-initiated telephone consultations to 
provide TA at two and four months following the work- 
shop. The Multimedia Connect program and facilitator 
training is a translation of the original, which features the 
same core elements [19], but replaces hard copy materials 
with a Web-based interface of translated interactive tools 
and video enhancements a facilitator uses as a 'road map' 
to lead participants through activities (see Figure 2). This 
version also dynamically monitors session progress (e.g., 
time spent on which activity, when, and by whom). The 
translation from manual-based intervention to multimedia 
version was based on design research methodology [56] 
and scaffolded learning theory [57], and is described in de- 
tail elsewhere [58,59]. The goals of this version were to 
simplify facilitation so that staff with a wider range of edu- 
cational and experiential levels, from peer volunteers to 
trained professionals, could use the program; to increase 
the couple's engagement with the materials; and to make 
it easy to rapidly deploy and disseminate to agencies via 
the Internet. 

Conceptual framework 

Study implementation was guided by social cognitive 
and scaffolded learning theories. Social cognitive theory 
[18,60] suggests implementation of a new program 



occurs when there is exposure to and motivation among 
agency staff to acquire the skills and resources to enact 
the program; the skills needed to implement the pro- 
gram are acquired through instruction, modeling, and 
rehearsal; and feedback and reinforcement are provided 
during the early phases of implementation. Reinforce- 
ment, in turn, is enhanced through scaffolded learning, 
which allows learners to revisit content as needed until 
they acquire new skills [57] . Building the multimedia fa- 
cilitator training into the same Web-based environment 
as the program and offering numerous retraining re- 
sources for facilitators present many opportunities for 
scaffolded learning. 

We provided scheduled program trainings once or 
twice per month on a rolling basis for groups of up 
to five agencies (15 - 20 staff participants) from the 
same condition (manual-based or Web-based). The 
study was longitudinal with repeated measurement of 
outcome variables at 6, 12, and 18 months post-training 
and TA. 

Technical assistance 

TA was offered in two ways. As part of each condition, 
TA was initiated by the research team to each agency at 
two and four months post-program training. During 
these hour-long conference calls, participants were asked 
a series of questions regarding their efforts to implement 
the program and asked questions regarding any barriers 
or issues faced. The research team provided support and 
problem-solving strategies to each issue or question raised. 
In addition to this TA as part of the process, we also en- 
couraged all participants to call or email at any time 
throughout the trial if they had any need for support re- 
lated to implementation of the program. These 'proactive' 
efforts on the part of participants were all recorded 
as they could also possibly influence the implementa- 
tion outcomes. 

As recommended in the literature, we employed a 
community advisory board (CAB) representing adminis- 
trators, clinicians, and clients from HIV prevention or- 
ganizations [61] to assure study protocols were relevant 
and appropriate for agency-based settings [62]. 

Assessment 

Prior to being trained in the program's implementation, 
individual staff participants completed informed consent 
and an online baseline assessment of demographic and 
organizational characteristics, descriptive variables, out- 
come variables and barriers to implementation. The pri- 
mary study outcomes were measures of implementation 
of the Connect program {e.g., delivered sessions), opera- 
tionalized as follows: the total number of couples with 
whom staff implemented Connect at each agency; the 
mean number of Connect sessions implemented by staff 
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Opening orientation screen for Multimedia Connect HIV Myth/Fact interactive game question 





HIV Myth/Fact interactive game answer 



Protective behaviors interactive game 





Satisfying Safer Sex arranger interactive game 
Figure 2 Screenshots of Multimedia Connect program. 



Sample "how to" screen to accompany each exercise 



at each agency whether staff implemented at least one 
session of Connect; and whether staff implemented a 
complete cycle of Connect in the prior six months. Early 
implementation of this type (also known as 'adoption') 
offers a window into early use and barriers to use of new 
programs [63,64]. In the original research design, out- 
comes were at the agency-level [65]. Barriers to imple- 
mentation were assessed at the 18-month follow-up by 
asking participants to 'check each item that may be a 
barrier to implementing CONNECT at your agency', 
asking for endorsement of as many as apply for them at 
their agency. Examples of potential barriers to endorse 
included 'lack of funds', 'inadequate staff, or 'inadequate 
training in intervention'. 
Figure 1 presents the study design and flow. 



Incentives 

Each participating agency received a laptop computer 
and $250 stipend to offset costs incurred due to partici- 
pation in training and interviews. Each participating staff 
received $25, $30, $35, and $40 in compensation for 
completion of baseline, 6, 12, and 18 month assessments, 
respectively. An additional incentive was that the training 
qualified the staff member for Certified Alcohol and Sub- 
stance Abuse Counselor continuing education credit. 

Sample size calculations and data analysis 

Power analyses-assuming alpha = 0.05, two-tailed testing, 
minimum 'small-to-medium' [66] effect size f 2 = 0.10, and 
variance inflation factors (VIFs) ranging from 1.2 to 1.5 
to account for the design effect resulting from use of 
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matched pairs during randomization-suggested that 80% 
power would be achieved with 80 agencies as the unit of 
analysis. 

Organizational characteristics and stratification factors 
were analyzed to check for balance and proper applica- 
tion of the random assignment process. Demographic 
data were analyzed for differences between participants 
from the two conditions. 

We relied on an intent-to-treat approach and we used 
generalized linear models with random effects for re- 
peated measures to estimate differences attributed to as- 
signment (Web-based = 1, Manual-based = 0) for four 
primary outcomes at the 6-, 12-, and 18-month follow 
up: the total number of couples with whom staff im- 
plemented Connect at each agency (negative binomial 
regression); the mean number of Connect sessions im- 
plemented by staff at each agency (linear regression); 
whether staff implemented at least one session of Connect 
(logistic regression); and whether staff implemented a 
complete cycle of Connect (logistic regression). Originally, 
the agency was the unit of analysis [65]. 

To be consistent with the agency level analyses, we 
used generalized linear models with random effects for 
repeated measures and agency to account for the clus- 
ters of individuals within agencies. Outcomes were oper- 
ationalized as above, but measured individually: the total 
number of couples with whom participant implemented 
Connect at his/her agency; sum of the number of Connect 
sessions implemented by participant at his/her agency; 
whether participant implemented at least one session of 
Connect; and whether participant implemented a com- 
plete cycle of Connect. This allowed accommodation for 
intraclass correlations (ICC) associated with reports from 
more than one facilitator at a given agency (range 0.16 — 
0.54). In addition, the agency level analysis alone would 
not account for the fact that an agency with more facilita- 
tors has higher reliability of estimates compared to those 
with fewer facilitators. 



Results 

Baseline equivalence of agencies 

The mean HIV/STI prevention budget for all agencies 
was $1,068,910; the median was $362,500 (range 0 to 
35,000,000). The mean number of full time prevention 
staff was 18; the median was 9 (range 0 - 216). Table 1 
describes organizational characteristics of participating 
agencies by condition. Despite variability, there were no 
significant differences in organizational characteristics at 
baseline by condition, suggesting the matched-pairs ap- 
proach for randomization yielded relatively balanced 
groups for the trial. The actual variance accounted for 
due to matched pairing procedure was <0.01, which in- 
dicates that the VIFs used to estimate power were con- 
servative statistically. 

Demographic characteristics of participants 

The pool of respondents included 253 agency staff (131 
Web-based and 122 manual-based), with an average of 
three participants per agency (range 1-6). Most were 
female and over 40 years of age. The majority of partici- 
pants were African American. Participants had an aver- 
age of seven years of experience in HIV services, and 
reported high levels of confidence in using both a com- 
puter and an Internet browser (Table 2). 

Implementation over time 
Agency level 

Over 18 months, 33% of Web-based and 48% of manual- 
based agencies implemented at least one session. Table 3 
presents the mean values for the primary outcomes by 
condition. As shown, for each six-month follow up period, 
the mean number of couples and/or sessions per agency is 
quite low. With the exception of 'completed at least one 
cycle, trends suggest that reported implementation out- 
comes were higher, but slowly decreased over time among 
manual-based agencies, while implementation was lower 
but increased over time among the Web-based agencies. 



Table 1 Characteristics of HIV service agencies by condition (N = 80) # 



Manual-based (Traditional) 



Web-based (Multimedia) 



Mean 



Median 



Range 



Mean 



Median 



Range 



HIV/STI prevention budget (in dollars) 


593,372 


280,175 


0-4,000,000 


1 ,544,449 


425,000 


0-35,000,000 


Full-time HIV prevention staff 


16 


/ 


1-147 


20 


10 


1-216 


Part time HIV prevention staff 


2 


1 


0-15 


3 


1 


0-20 


No. of clients receiving HIV prevention 


7,207 


1,192 


20-81,500 


3,483 


1,200 


50-40,000 


No. of clients receiving HIV prevention 


7,207 


1,192 


20-81,500 


3,483 


1,200 


50-40,000 


No. of clients receiving multi-session services 


543 


126 


0-4,000 


472 


95 


0-8,000 


% of heterosexual clients 


93 


80 


9-95 


76 


80 


25-95 


No. computers available for implementation* 


8 


3 


1-111 


8 


3 


0-200 


No. DEBIs offered at baseline* 


1 


1 


0-7 


1 


2 


0-6 



#rounded to whole numbers. 

individual level data from baseline assessment aggregated by agency. 
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Table 2 Sociodemographic characteristics for participants 
(N = 253) 



Manual-based Web-based 
(Traditional) (N = 1 31 ) (Multimedia) (n = 1 22) 



Sociodemographics 


Frequency 


Percentage 


Frequency 


Percent 


Sex 










Female 


89 


67.9 


92 


75.4 


Male 


42 


32.1 


30 


24.6 


Age 










18-29 


18 


13.7 


22 


18.0 


30-39 


35 


26.7 


31 


25.4 


40 and above 


78 


59.5 


69 


56.6 


Marital status 










Married 


45 


34.4 


36 


29.5 


Separated, Divorced, 
Widowed 


28 


21.3 


28 


23.0 


Single 


58 


44.3 


58 


47.5 


Sexual orientation 










Heterosexual 


110 


84.0 


99 


81.1 


Gay 


10 


7.6 


12 


9.8 


Lesbian 


6 


4.6 


6 


4.9 


Bisexual 


3 


2.3 


0 


0 


Other 


1 


0.8 


1 


.8 


Race-Ethnicity 










African American 


57 


43.5 


47 


38.5 


Hispanic 


45 


34.4 


47 


38.5 


White or Caucasian 


28 


21.4 


27 


22.1 


Caribbean, West 
ndian 


9 


6.9 


10 


8.2 


Asia, SE Asia, Pac Isla. 


7 


5.3 


3 


2.5 


Amer. Indian/ 
Alaska Nat. 


1 


0.8 


1 


0.8 


Other. 


4 


3.1 


1 


0.8 


Born in 50 U.S. states 


97 


74.0 


99 


81 .1 


Preferred language 










English 


115 


87.8 


106 


86.9 


Spanish 


9 


6.9 


12 


9.8 


Other 


7 


5.4 


4 


3.2 


Education 










High School or GED 


17 


13.0 


23 


18.9 


College Technical 


64 


48.9 


49 


40.2 


Graduate school 


50 


38.2 


50 


41.0 




Mean 


Range 


Mean 


Range 


Yrs in HIV/STI service 


4.6 


.5-13 


7.1 


.5-13 


Yrs at agency 


4.7 


.5-13 


2.9 


.5-13 


Computer confidence 


8.6 


2-10 


9 


3-10 


Browser comfort 


8.6 


0-10 


9 


1-10 



A x 2 test of independence found no statistically significant (p < 0.05) 
association with assignment. 



Participant level 

Over 18 months, 17% of participants assigned to Web- 
based agencies and 26% of staff assigned to manual-based 
agencies implemented at least one session. Consistent 
with agency level outcomes, trends suggest that reported 
implementation outcomes were higher, but slowly de- 
creased over time among manual-based agencies, while 
implementation was lower but increased over time among 
the Web-based agencies. 

Hypothesis testing: primary outcomes 
Agency level 

Table 4 demonstrates findings of multivariate analyses 
over time. At the agency level, only hypothesis one of 
four yielded significant findings. 

Outcome one: The outcome estimates demonstrate 
that, on average, the staff from Web-based agencies im- 
plemented the intervention to 35% fewer couples than 
those at manual-based agencies (CI, 0.13 - 0.94). 

Outcomes two, three, and four: There were no differ- 
ences found between conditions on the average number of 
sessions completed per facilitator (b = -0.31; CI, -0.068 - 
0.06); the likelihood to implement at least one session 
(OR = 0.30; CI, 0.06 - 1.46); or the likelihood to complete 
at least one cycle of Connect (OR = 0.80; CI, 0.12 - 5.55). 

Individual level 

At the participant level, no significant differences were 
found between groups for any of the four outcomes. The 
outcome estimates shown in Table 4 demonstrate no dif- 
ferences between participants at Web-based agencies com- 
pared to those at manual-based agencies on average 
number of couples to whom they provided sessions (IRR = 
0.45; CI, 0.14 - 1.45); number of sessions completed per fa- 
cilitator (b = -0.025; CI, -0.057 - 0.08); the likelihood to im- 
plement at least one session (OR = 0.34; CI, 0.06 - 1.46); or 
the likelihood to complete at least one cycle of Connect 
(OR = 0.71; CI, 0.13 - 3.84). 

Technical assistance 

Attendance for the investigator-initiated TA calls at two 
and four months was high (see Figure 1), however, few 
participating staff called the research team for TA at 
other times. Over the 18-month post-program period we 
received a total of only 28 requests, including a combin- 
ation of phone calls from participants in both groups 
and emails from the 'help' button of the Web-based 
program. 

Barriers to implementation 

The three most frequently reported barriers to implemen- 
tation were economy/funding issues (47% and 53%), re- 
cruitment of couples (46% and 51%), and staff turnover 
(34.9% and 22%) among manual-based and Web-based 
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Table 3 Implementation over time by condition 





Agency level 






Individual 


level 




6-Month 
(n = 80) 


1 2-Month 
(n = 80) 


18-Month 
(n = 78) 


6-Month 
(n = 228) 


12-Month 
(n = 222) 


18-Month 
(n = 211) 


Number of couples per facilitator: Mean (SD) 














Manual-based 


1 .38 (4.79) 


/. 15* (2.53) 


1.11 (3.34) 


0.46 (2.00) 


0.40** (1.11) 


0.40 (1.49) 


Web-based 


0.53 (1.47) 


0.25* (0.67) 


0.98 (2.90) 


0.19 (0.91) 


0.09** (0.38) 


0.37 (1.36) 


Overall 


0.95 (3.55) 


0.70 (1.89) 


1.04 (3.10) 


0.33 (1.58) 


0.25 (0.85) 


0.38 (1.42) 


Mean of sessions per facilitator: Mean (SD) 














Manual-based 


0.49 (1.27) 


0.69* (1.31) 


0.48 (1.21) 


0.41 (1.24) 


0.60* (1.52) 


0.50 (1.51) 


Web-based 


0.23 (0.61) 


0.1 5* (0.39) 


0.37 (0.80) 


0.24 (0.99) 


0.19* (0.83) 


0.44 (1.36) 


Overall 


0.36 (1.00) 


0.42 (1.00) 


0.42 (1.01) 


0.33 (1.13) 


0.40 (1.25) 


0.47 (1.43) 


Implemented at least one session: n (%) 














Manual-based 


1 2 (30%) 


14* (35%) 


1 0 (26%) 


17 (14%) 


22** (19%) 


13 (12%) 


Web-based 


8 (20%) 


6* (15%) 


9 (23%) 


9 (8%) 


7** (7%) 


12 (11%) 


Overall 


20 (25%) 


20 (25%) 


1 9 (24%) 


26 (11%) 


29 (13%) 


25 (12%) 


Completed at least one cycle: n (%) 














Manual-based 


2 (5%) 


4 (1 0%) 


6 (16%) 


3 (3%) 


6 (5%) 


9 (8%) 


Web-based 


3 (8%) 


1 (3%) 


6 (15%) 


3 (3%) 


1 (1%) 


8 (8%) 


Overall 


5 (6%) 


5 (6%) 


12 (15%) 


6 (3%) 


7 (3%) 


1 7 (8%) 



*p < 0.05; **p < 0.01 between conditions at each time point. 



conditions respectively. No significant differences between 
conditions were found on any reported barrier. 

Discussion 

This study is the first to examine implementation of a 
couple-based HIV prevention intervention, and the first 
to test competing modalities for implementation of a 
behaviorally- based HIV prevention program (in this 
case, a CDC DEBI) among HIV services agencies. Find- 
ings did not support our hypotheses, demonstrating in- 
stead that whether measured at the level of the agency 
or the participants, staff trained on the Web-based pro- 
gram did not demonstrate significantly higher implemen- 
tation compared to those trained on the manual-based 
approach. Agency level outcomes did demonstrate higher 
mean number of couples with whom staff at manual- 
based agencies worked compared to Web-based agencies, 
however, these findings were not significant at the individ- 
ual level. Findings also show overall low rates of imple- 
mentation, but implementation rates that are consistent 
with the landmark study on which the CDC DEBI ap- 
proach is based [18]. Our discussion therefore focuses 
on potential explanations for these findings and recom- 
mendations for future research on couple-based program 
implementation. 

As a dissemination trial, commitment from agencies to 
implement Connect was not an eligibility criterion. Fur- 
thermore, we did not offer additional resources or incen- 
tives for implementation. Our goal was to observe how 



agencies managed the process of manual or Web-based 
implementation with their clients post-training and TA. 
We did observe implementation: staff at 33% of Web- 
based and 48% of traditional agencies implemented ses- 
sions over 18 months; 17% of participants assigned to 
Web-based agencies and 26% of staff assigned to trad- 
itional agencies implemented sessions over 18 months. 
A smaller number implemented multiple sessions and 
cycles and was able to sustain use at low levels. The eco- 
nomic recession that began in 2008 had a devastating ef- 
fect on HIV services agencies in New York State — some 
lost as much as 30% of their funding within three to six 
months. Because most participants reported their agen- 
cies were already implementing at least one DEBI, ab- 
sent additional incentives or funding, more staff may 
have felt more comfortable implementing the traditional 
version of Connect, which did not require access to a 
computer or wireless Internet. Of note is that these low 
percentages are consistent with and only slightly lower 
than those reported in Kelly et al. [18,19], showing 
'adoption' or 'offers of service' rates of between 36% and 
60% of agencies participating in an implementation trial 
for another DEBI, the Peer Opinion Leader (POL) 
model. In this prior study the outcome was defined as 
'offers of service' and not actual implementation, leaving 
us unable to examine whether actual rates of program 
'use' are comparable. 

The expected benefits of a Web-based program not- 
withstanding, implementation rates may reflect a time of 
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Table 4 Estimated effect of assignment to web-based versus manual-based connect at both agency and individual/ 
participant levels: longitudinal multilevel analysis 





Agency level (N=80) 1 




Individua 


I level (N=253) 2 






Negative 


Binomial Regression 


Estimates 


Negative E 


iinomial Regression 


Estimates 




IRR 


CI 


P 


IRR 


CI 


P 


Total number of couples 














Average of 3 waves 


0.35* 


0.13 - 0.94 


0.038 


0.45 


0.14- 1.45 


0.181 


At 6-month only 


0.27* 


0.09 - 0.84 


0.024 


0.30 


0.08 -1.10 


0.070 


At 12-month only 


0.36* 


0.13 - 0.99 


0.047 


0.43 


0.13 - 1.40 


0.163 


At 18-month only 


0.49 


0.16 - 1.55 


0.226 


0.62 


0.18 - 2.20 


0.464 




Linear Regression Estimates 




Linear Reg 


ression Estimates 






b 


CI 


P 


b 


CI 


P 


Mean number of sessions completed 














Average of 3 waves 


-0.31 


-0.68 - 0.06 


0.101 


-0.25 


-0.57 - 0.08 


0.137 


At 6-month only 


-0.38 


-0.80 - 0.04 


0.079 


-0.29 


-0.66 - 0.08 


0.124 


At 12-month only 


-0.31 


-0.68 - 0.06 


0.101 


-0.25 


-0.58 - 0.08 


0.133 


At 18-month only 


-0.25 


-0.67 - 0.18 


0.257 


-0.21 


-0.59 - 0.17 


0.279 




Logistic Regression Estimates 




Logistic Re 


.'gression Estimates 






OR 


CI 


P 


OR 


CI 


P 


Implement at least 1 session 














Average of 3 waves 


0.30 


0.06 - 1 .46 


0.136 


0.34 


0.09 - 1 .26 


0.105 


At 6-month only 


0.23 


0.03 - 1 .50 


0.124 


0.23 


0.05 - 1 .09 


0.063 


At 12-month only 


0.30 


0.06 - 1 .47 


0.137 


0.34 


0.09 - 1 .25 


0.104 


At 18-month only 


0.40 


0.06 - 2.57 


0.332 


0.49 


0.11 - 2.22 


0.354 


Completed at least 1 cycle 














Average of 3 waves 


0.80 


0.12 - 5.55 


0.819 


0.71 


0.13 - 3.84 


0.691 


At 6-month only 


1.06 


0.05 - 21.1 


0.972 


0.67 


0.05 - 8.31 


0.756 


At 12-month only 


0.82 


0.09 - 7.54 


0.861 


0.66 


0.10-4.44 


0.667 


At 18-month only 


0.64 


0.06 - 7.32 


0.717 


0.65 


0.09 - 4.74 


0.666 


*p<0.05. 
Note: 

1 . Measurements at the agency level: 

a. Number of couples at the agency level = 


sum(# couple a 


facilitator reported). 











b. Mean number of sessions completed at the agency level = sum{mean number of sessions completed by a facilitator)/(number of facilitator in an agency). 

c. Implemented at least I session at the agency level = at least one facilitator reported implementing at least one session in an agency. 

d. Completed at least I cycle at the agency level = at least one facilitator reported completing at least one cycle in an agency. 
2. Measurements at the individual level: 

a. Number of couples at the individual level = sum(# couple a facilitator reported). 

b. Mean number of sessions completed at the individual level = sum(number of sessions completed with a couple)/(number of couples). 

c. Implemented at least I session at the individual level = facilitators reported implementing at least one session. 

d. Completed at least I cycle at the individual level = facilitators reported completing at least one cycle. 



low or dwindling resources or of considerable organiza- 
tional disruption in services provision due to the reces- 
sion and related cuts. Examination of data related to 
changes in organizational budgets over time may further 
inform interpretation of findings. During the study per- 
iod four agencies (three manual-based and one Web- 
based) chose to incorporate Connect into competing 
grant renewals from New York State or the CDC to con- 
tinue implementation. These were successful and post 
18-month follow-up, these agencies continue imple- 
mentation. Future trials may consider ways to offer short 



term resources in support of implementation or increase 
time spent during training sessions to how best to inte- 
grate new programs into existing service delivery or re- 
imbursement mechanisms. 

Our expectation that facilitators would use a Web- 
based strategy and facilitate a couple-based intervention 
at once may have been unrealistic and may also explain 
the low usage of the Web-based Connect among our 
agency participants. With few exceptions, the facilitators 
had never worked with couples. Enthusiasm during pro- 
gram trainings to implement a couple-based program 
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suggested individual level commitment to implement 
the program. However, many expressed concerns about 
how they would manage couple-related interventions at 
their agencies. While Connect activities are simplified 
for easy implementation and are intended to be delivered 
by a wide range of facilitators, implementing a couple- 
based intervention presents a unique set of challenges 
[13,14]. Staff may have felt unduly challenged by the sim- 
ultaneous demands of attending to both the new elements 
of a Web-based intervention and the needs of the dyad. 
We did not include adequate measures to examine this in- 
terpretation, so strongly recommend that future imple- 
mentation trials incorporate measures sensitive to both 
the innovation of the program and the approach to imple- 
mentation. Given the increasing attention to both couple- 
based and Web-based HIV prevention and the potential 
advantages cited [13,14] though not borne out in this 
study, future implementation trials might incorporate add- 
itional training and supervision for any new or novel com- 
ponent of implementation, particularly in the beginning 
stages. 

Technology management issues including staff com- 
fort and concerns about client data security and confiden- 
tiality may also account for our finding that Web-based 
implementation did not surpass manual-based implemen- 
tation. Our choice of computer model and browser may 
have been unfamiliar to staff or different from the stand- 
ard agency set-up to which they were accustomed. Most 
participating agencies employed IT staff or consultants 
that must approve or conduct any changes to agency- 
related computing; this sometimes delayed downloading 
and installation of required software and may have created 
real or imagined barriers to implementation. Further, des- 
pite almost six months of beta-testing to ensure there 
were no major issues with accessibility or utility of the 
program, 'bugs' in the system were reported during TA 
calls by some staff who tried the program early on. To- 
gether, these issues may have impeded a smooth transition 
to regular use of the program, or worse, led them to de- 
cide not to use the program at all. To address such issues 
in future trials, we recommend creating any Web-based 
program with support for a range of hardware and soft- 
ware (i.e., various browsers). It would also be important to 
create an understanding with adopting agency staff that 
reporting glitches and bugs is important so that issues can 
be corrected quickly. 

During the training for the Web-based Connect, many 
participants asked about the security of the program. Al- 
though we discussed the steps taken to ensure confi- 
dentiality of all program data, participating staff cited 
Health Information Portability and Accountability Act 
(HIPAA)-related concerns regarding access to the infor- 
mation gathered on couples engaged in the Web-based 
program activities. The program did not save identifiable 



data, rendering HIPAA concerns unwarranted. Future re- 
search should provide follow-up — with support from 
agency administrative staff — to clarify with staff that any 
programming adopted at the agency provides adequate 
protection of client confidentiality. 

Reported barriers to adoption across conditions 

Consistent with outcomes, reported barriers to implemen- 
tation did not differ by condition. The top barriers reported 
were lack of funding and staff turnover, consistent with 
existing literature on program adoption [1,20,21,41,67-70]. 
The next most frequendy reported barrier was client re- 
cruitment, which suggests the definition of 'couple' may 
need more attention in the training sessions. During many 
TA calls, participants noted their agency 'did not serve 
couples' and therefore they could not offer a couple-based 
program. When probed, staff members reported serving 
individuals who were partnered. However, a shift in ideol- 
ogy appears to be needed in order for agency staff to see 
'couples' as a unit of analysis that integrates with other pro- 
gram units ( e.g., 'individual', 'group') for purposes of do- 
cumentation and billing. Researchers need to work with 
agencies to identify how best to integrate new couple- 
based EBIs into new or existing services [13] and billing 
mechanisms. 

Qualitative data collected in in-depth interviews and 
site visits to 10% of participating agencies should yield 
additional insights to further interpret these findings. 

Technical assistance 

Our finding of low TA requests is not a new pheno- 
menon. Veniegas et al. (2009) [21] report none of the 34 
staff members interviewed among CDC-funded agencies 
in the Los Angeles area implementing EBIs (in a trad- 
itional format) made a single request for TA. Neverthe- 
less, this finding raises critical questions regarding how 
to best meet the needs of new implementers, who would 
naturally have questions and need assistance, but are not 
yet requesting the TA available to them, and which em- 
pirical data suggest will improve implementation. What 
are the barriers to TA use? How do we motivate staff to 
request TA? Kegeles et al. (2012) [25], in an implemen- 
tation project for MPowerment, another CDC DEBI, 
provided proactive TA every two weeks. After six months, 
requests for TA increased and then remained high for as 
long as two years [25]. Such a strategy may strengthen 
both manual- and Web-based implementation approaches 
and we recommend offering more targeted proactive TA 
during early stages of all adoption and implementation 
efforts. 

Limitations 

Findings should be considered in light of several limita- 
tions. First, findings can only be interpreted to indicate 
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that we found no evidence for the relationship, not evi- 
dence for lack of a relationship. The study was not pow- 
ered as an equivalence trial, but rather to assess whether 
the Web-based approach was superior to the manual- 
based approach. The noted technology challenges are 
more anecdotal lessons learned to influence future trials, 
and cannot be interpreted to suggest that they were re- 
sponsible for lower implementation of the Web-based 
program. Data were based on self-report only, which in 
some cases can lead to poor recall, and which can reduce 
confidence in findings. Operationalizing implementation 
is challenging; any single measure often inadequate [71]. 
While we expanded our definition and examination of 
outcomes beyond the existing model of HIV prevention 
implementation [18], their reliability may still be weak. A 
number of additional assessment questions {e.g. partici- 
pant experience working with couples) may have streng- 
thened analysis of findings. Finally, the loss of funds to 
agencies due to the recession during this trial makes it im- 
possible to apply the findings to a more typical economic 
climate. 

Conclusions 

Despite limitations, the study makes critical contributions 
to HIV prevention implementation science. Findings show 
the feasibility of training on and implementing both a 
manual-based and a Web-based version of a couple-based 
DEBI (or other evidence-based, manualized, behavioral 
intervention), and the enthusiasm of HIV services organi- 
zations and their staff to participate in implementation tri- 
als in the absence of strong economic incentives. In 
particular, agencies embraced the opportunity to expand 
service provision to include couple-based and Web-based 
programming. Couples-based approaches for HIV testing 
and counseling, as well as for more intensive primary and 
secondary prevention efforts are increasing. Implementa- 
tion methods must be identified to best bring them to 
scale. Increasing interest in both of these areas of HIV 
prevention efficacy studies suggest the need for more 
careful study to inform rapid and efficient implementation 
of efficacious programs. 

Study findings raise questions regarding the implemen- 
tation of such programming in the absence of dedicated 
funding, however, and of barriers. Our interpretation of 
findings suggest that either a very poor economy, which 
rapidly reduced resources at participating agencies and led 
to low overall implementation, and (for one outcome) 
higher manual-based implementation; or else the com- 
bined challenge of using both a web-based curriculum 
and also implementing couple-based programming for the 
first time, may have contributed to the lack of hypothe- 
sized outcomes. Findings point to several important issues 
for future implementation research in HIV prevention, 
with couple-based programs, and/or with Web-based 



programming. Web-based programming should be built 
to support a range of browsers and beta-tested thoroughly 
prior to trial initiation. During trials, participants should 
receive more frequent proactive TA to ensure movement 
from the pre-implementation to implementation stages, 
and more time and assistance to agencies to integrate new 
programs into their existing program structure and reim- 
bursement mechanisms. This should enhance support 
from administrators and smooth transitions for line wor- 
kers managing multiple responsibilities. Support to 
agencies in identifying external funding for new pro- 
gram implementation should further reduce barriers to 
implementation. Future dissemination studies should also 
consider not only a comparison of mediums, but also the 
features of Web-based programming that can best support 
facilitators [72,73]. The task ahead is to strengthen train- 
ings for implementation to include not just information 
regarding the innovation or program, but the implementa- 
tion procedures and expectations (including access to re- 
sources) to best prepare agencies for the transition to 
couple- and Web-based approaches and ongoing support, 
and to identify whether, how, and when it may be efficient 
to use human-delivered and Web-based components of 
programming to achieve implementation. Future work 
may also consider hybrid interventions introduced in face- 
to-face single sessions by medical and mental health prac- 
titioners, and followed by self-administered, Web-based 
sessions accessed by participants or clients and their part- 
ners or other family members. Such approaches are likely 
given the present state of the economy, healthcare re- 
form and changes in reimbursement for healthcare in 
the United States. In the end, the key factors barring 
most agencies post-training from successfully imple- 
menting the program remain unclear. We hope these 
may be borne out in ongoing analysis of the dataset, 
including qualitative data from agency site visits. 

In the years since the initiation of this trial, the policy 
and best practice of HIV prevention and the use of 
digital technologies have changed dramatically. Interven- 
tions are being shortened to one or two sessions in the 
name of feasibility, economy, and practical dissemination 
[7]. Interventions are also now primarily targeted to in- 
dividuals living with HIV and their intimate partners [6] . 
The Web-based program developed and implemented in 
this trial is the prototype for a number of HIV preven- 
tion programs currently being tested for efficacy and ef- 
fectiveness [74,75]. The program's software 'backbone' 
can be repurposed for future use in the dissemination of 
similar, activity-based, multi-session programs. Web- 
based programming will take on even more importance 
with the continuing economic challenges to agencies 
and reduced time for prevention services. Building on 
small successes and lessons learned here should streng- 
then future efforts. 
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